Electronic data generation methods

ABSTRACT

An electronic method for generating group data indicative of the collective response of at least a subset of respondents to at least one portion of a time-varying stimulus the method comprising: receiving responses to the time-varying stimulus from respective ones of a plurality of respondent devices, each response including data indicative of a response of a respondent associated with the respective respondent device to the time-varying stimulus; filtering the responses to generate a filtered response data set that excludes responses outside of a timing tolerance relative to a portion of the time-varying stimulus to which the group data is intended to relate; and processing at least a subset of the filtered response data set to generate the group data.

RELATED APPLICATION

This application is based on and claims the benefit of the filing andpriority dates of AU application no. 2011902840 filed 15 Jul. 2011, thecontent of which as filed is incorporated herein by reference in itsentirety.

FIELD

The invention relates to electronic data generation methods, an outputdata generation system, a respondent device, and an electronic method ofcontrolling communication of response data.

BACKGROUND

A number of techniques have been used to determine the opinion ofrespondents to a stimulus at a point in time. Often these techniquesinvolve asking particular qualitative or quantitative questions and thenlogging and consolidating the individual responses. The responses tothese questions are generally “static”. Dynamically measuring responseto a dynamic stimulus such as a live debate, a video, or audio hastypically been undertaken by asking questions at a point in time. Oncedata is gathered, it is usually processed to generate output data. Theoutput data may be displayed visually.

There is a need for alternative techniques for use generating outputdata that captures the responses of respondents.

SUMMARY

In a first aspect, the invention provides an electronic method forgenerating group data indicative of the collective response of at leasta subset of respondents to at least one portion of a time-varyingstimulus the method comprising:

-   -   receiving responses to the time-varying stimulus from respective        ones of a plurality of respondent devices, each response        including data indicative of a response of a respondent        associated with the respective respondent device to the        time-varying stimulus;    -   filtering the response data to generate a filtered response data        set that excludes responses outside of a timing tolerance        relative to a portion of the time-varying stimulus to which the        group data is intended to relate; and    -   processing at least a subset of the filtered response data set        to generate the group data.

In an embodiment, the method comprises outputting the group data to adisplay generation system arranged to process the group data to generatea display of group data.

In an embodiment, the method comprises processing the at least a subsetof the filtered response data set to generate a display of the groupdata.

In an embodiment, the collective response of respondents

-   -   is to be displayed over a period of time such that respondents        provide responses related to plural portions of the time-varying        stimulus, and the method comprises repeatedly filtering the        responses to generate plural filtered response data sets within        a timing tolerance of respective ones of plural portions of the        time-varying stimulus such that each response data set is        comprised of responses within the timing tolerance whereby a        first response from at least one specific respondent device is        included in at least one filtered response data set and a second        response from the at least one specific respondent device is        excluded from at least one filtered data set.

In an embodiment, the response data includes time data indicative of atime at which the response data was generated and the time data ischecked when filtering the response data to determine whethercommunication of the response data involved a delay greater than thetiming tolerance.

In an embodiment, the method comprises receiving an external computingsystem identifier of one of the respondents, and identifying one or moreof the other respondents based on the external computing systemidentifier of the respondent.

In an embodiment, identifying the other respondents comprises retrievingdata enabling an identification of respondents from an externalcomputing system using the external computing system identifier of therespondent.

In an embodiment, the method comprises:

-   -   filtering the responses to generate a respondent response data        set that excludes responses not received from the identified        other respondents;    -   generating message data based on the respondent response data        set; and    -   transmitting the message data to the external computing system        using the external computing system identifier of the        respondent.

In an embodiment, the message data comprises one or more metrics,scores, or metrics and scores.

In an embodiment, the external computing system provides a socialnetworking website.

In a second aspect, the invention provides an output data generationsystem for generating group data indicative of the collective responseof at least a subset of respondents to at least one portion of atime-varying stimulus, the system comprising:

-   -   a communication interface arranged to receive responses to the        time-varying stimulus from respective ones of a plurality of        respondent devices, each response including data indicative of a        response of a respondent associated with the respective        respondent device to the time-varying stimulus;    -   a data processor arranged to filter the response data to        generate a filtered response data set that excludes responses        outside of a timing tolerance relative to a portion of the        time-varying stimulus to which the group data is intended to        relate and process at least a subset of the filtered response        data set to generate the group data.

Thus, some embodiments of the invention enable responses with excessivecommunication lag to be excluded from the displayed collective response.

In an embodiment, the communication interface is arranged to output thegroup data to a display generation system arranged to process the groupdata to generate a display of group data.

In an embodiment, the data processor is arranged to process the at leasta subset of the filtered response data set to generate a display of thegroup data.

In an embodiment, the collective response of respondents is to bedisplayed over a period of time such that respondents provide responsesrelated to plural portions of the time-varying stimulus; and

-   -   the data processor is arranged to repeatedly filter the        responses to generate plural filtered response data sets within        a timing tolerance of respective ones of plural portions of the        time-varying stimulus such that each response data set is        comprised of responses within the timing tolerance,    -   whereby a first response from at least one specific respondent        device is included in at least one filtered response data set        and a second response from the at least one specific respondent        device is excluded from at least one filtered data set.

In an embodiment, the response data includes time data indicative of atime at which the response data was generated and the time data ischecked when filtering the response data to determine whethercommunication of the response data involved a delay greater than thetiming tolerance.

In an embodiment, the communication interface is arranged to receive anexternal computing system identifier of one of the respondents, and thedata processor is arranged to identify one or more of the otherrespondents based on the external computing system identifier of therespondent.

In an embodiment, the data processor identifies the other respondents bycontrolling the communication interface to retrieve data enabling anidentification of respondents from an external computing system usingthe external computing system identifier of the respondent.

In an embodiment, the data processor is arranged to:

-   -   filter the responses to generate a respondent response data set        that excludes responses not received from the identified other        respondents;    -   generate message data based on the respondent response data set;        and    -   control the communication interface to transmit the message data        to the external computing system using the external computing        system identifier of the respondent.

In an embodiment, the message data comprises one or more metrics,scores, or metrics and scores.

In an embodiment, the external computing system provides a socialnetworking website.

In a third aspect, the invention provides an electronic method ofcontrolling response data communicated to a data processing system froma respondent device, the method comprising:

-   -   receiving, via an input device of a respondent device, input        indicative of the respondent's responses to at least one portion        of a time-varying stimulus;    -   monitoring the input device to identify any portions of the        input that meet an attention criterion; and    -   communicating response data based on only the portions of the        input that meet the attention criterion from the respondent        device to a data processing system.

In an embodiment, the input device comprises a touch screen operable tocontrol an input interface displayed on a display associated with thetouch screen, and the attention criterion is that the respondent hasmaintained contact with the touch screen.

In an embodiment, the attention criterion is that the respondent hasaltered the input within a defined time period.

In an embodiment, the time-varying stimulus comprises a transmission tothe respondent.

In an embodiment, the method comprises displaying the transmission onthe respondent device.

In an embodiment, the method comprises:

-   -   receiving an external computing system identifier of the        respondent; and    -   upon receiving an input indicative of a response of the        respondent,        -   generating message data; and        -   communicating the message data to an external computing            system using the external computing system identifier of the            respondent.

In an embodiment, the message data includes information indicative of orrelated to the time-varying stimulus.

In an embodiment, the information is a description of the time-varyingstimulus.

In an embodiment, the external computing system provides a socialnetworking website.

In a fourth aspect, the invention provides a respondent device forcontrolling response data communicated to a data processing system, therespondent device comprising:

-   -   an input device operable by a respondent to input responses to        at least one portion of a time-varying stimulus;    -   an input monitor arranged to monitor the input device to        identify any portions of the input that meet an attention        criterion; and    -   a communication module arranged to communicate response data        based on only the portions of the input that meet the attention        criterion from the respondent device to a data processing        system.

In an embodiment, the input device comprises a touch screen operable tocontrol an input interface displayed on a display associated with thetouch screen, and the attention criterion is that the respondent hasmaintained contact with the touch screen.

In an embodiment, the attention criterion is that the respondent hasaltered the input within a defined time period.

In an embodiment, the time-varying stimulus comprises a transmission tothe respondent.

In an embodiment, the device comprises a display arranged to display thetransmission.

In an embodiment, the input device is operable by the respondent toinput an external computing system identifier of the respondent, and thedevice is arranged to, upon an input indicative of a response of therespondent being input, generate message data and communicate themessage data to an external computing system using the externalcomputing system identifier of the respondent.

In an embodiment, the message data includes information indicative of orrelated to the time-varying stimulus.

In an embodiment, the information is a description of the time-varyingstimulus.

In an embodiment, the external computing system provides a socialnetworking website.

In a fifth aspect, the invention provides an output data generationsystem comprising:

-   -   a data processing system; and    -   a plurality of respondent devices, each respondent device        comprising:        -   an input device operable by a respondent to input responses            to at least one portion of a time-varying stimulus;        -   an input monitor arranged to monitor the input device to            identify any portions of the input that meet an attention            criterion; and        -   a communication module arranged to only communicate response            data based on only the portions of the input that meet the            attention criterion from the respondent device to the data            processing system,    -   the data processing system arranged to process the communicated        response data from at least a subset of the plurality of        respondent devices to generate group output data indicative of a        collective response to the time-varying stimulus.

In an embodiment, the input device of each respondent device comprises atouch screen operable to control an input interface displayed on adisplay associated with the touch screen, and the attention criterion isthat the respondent has maintained contact with the touch screen.

In an embodiment, the attention criterion with respect to eachrespondent device is that the respondent has altered the input within adefined time period.

In an embodiment, the time-varying stimulus comprises a transmission tothe respondent.

In an embodiment, each respondent device comprises a display arranged todisplay the transmission.

In an embodiment, the input monitor of each respondent device isoperable by a respective respondent to input an external computingsystem identifier of the respondent, and each respondent device isarranged to, upon an input indicative of a response of a respectiverespondent being input, generate message data and communicate themessage data to an external computing system using the externalcomputing system identifier of the respondent.

In an embodiment, the message data includes information indicative of orrelated to the time-varying stimulus.

In an embodiment, the information is a description of the time-varyingstimulus.

In an embodiment, the external computing system provides a socialnetworking website.

In a sixth aspect, the invention provides computer program code whichwhen executed is arranged to:

-   -   enable a respondent to input responses to at least one portion        of a time-varying stimulus;    -   monitor input indicative of the respondent's responses to        identify any portions of the input that meet an attention        criterion; and    -   communicate response data based on only the portions of the        input that meet the attention criterion to a data processing        system.

In an embodiment, the respondent's responses are input using a touchscreen, and the attention criterion is that the respondent hasmaintained contact with the touch screen.

In an embodiment, the attention criterion is that the respondent hasaltered the input within a defined time period.

In an embodiment, the time-varying stimulus comprises a transmission tothe respondent.

In an embodiment, the computer program code when executed is arranged tocontrol a display to display the transmission.

In an embodiment, the computer program code when executed is arrangedto:

-   -   receive an external computing system identifier of the        respondent; and    -   upon receiving an input indicative of a response of the        respondent,        -   generate message data; and        -   communicate the message data to an external computing system            using the external computing system identifier of the            respondent.

In an embodiment, the message data includes information indicative of orrelated to the time-varying stimulus.

In an embodiment, the information is a description of the time-varyingstimulus.

In an embodiment, the external computing system provides a socialnetworking website.

In a seventh aspect, the invention provides an electronic method forgenerating data for display based on a time-varying stimulus stored in amemory in association with descriptive metadata corresponding to atleast one part of the time-varying stimulus, the method comprising

-   -   outputting the time-varying stimulus to one or more respondents;    -   receiving response data of the one or more respondents to the        time-varying stimulus; and    -   generating a time series of output data for display based on the        received response data and the descriptive metadata.

In an embodiment, there are at least two different items of descriptivemetadata corresponding to different parts of the time-varying stimulus,and the method comprises generating different outputs in dependence onthe different items of descriptive metadata.

In an embodiment, the method comprises generating the time series ofoutput data automatically upon determining that sufficient response datahas been received.

In an embodiment, the method comprises generating the time series ofoutput data automatically upon conclusion of output of the time-varyingstimulus to a plurality of respondents.

In an embodiment, the method comprises processing the response data ofat least a subset of respondents in conjunction with respondent profiledata of the respondents to generate at least one metric that complementsthe time series of output data.

In an embodiment, the method comprises receiving an external computingsystem identifier of one of the respondents, and identifying one or moreof the other respondents based on the external computing systemidentifier of the respondent.

In an embodiment, identifying the other respondents comprises retrievingdata enabling an identification of respondents from an externalcomputing system using the external computing system identifier of therespondent.

In an embodiment, the method comprises:

-   -   filtering the time series of output data to generate a        respondent time series of output data set that excludes        responses not received from the identified other respondents;    -   generating message data based on the time series of output data        set; and    -   transmitting the message data to the external computing system        using the external computing system identifier of the        respondent.

In an embodiment, the external computing system provides a socialnetworking website.

In an eighth aspect, the invention provides an output data generationsystem comprising:

-   -   a memory storing time-varying stimulus data in association with        descriptive metadata corresponding to at least one part of the        time-varying stimulus, the output data generation system        arranged to:    -   control output of the time-varying stimulus to one or more        respondents;    -   receive response data of the one or more respondents to the        time-varying stimulus; and    -   generate a time series of output data for display based on the        received response data and the metadata.

In an embodiment, there are at least two different items of descriptivemetadata corresponding to different parts of the time-varying stimulus,and the system is arranged to generate different outputs in dependenceon the different items of descriptive metadata.

In an embodiment, system is further arranged to generate the time seriesof output data automatically upon determining that sufficient responsedata has been received.

In an embodiment, system is further arranged to generate the time seriesof output data automatically upon conclusion of output of thetime-varying stimulus to a plurality of respondents.

In an embodiment, system is further arranged to process the responsedata of at least a subset of respondents in conjunction with respondentprofile data of the respondents to generate at least one metric thatcomplements the time series of output data.

In an embodiment, system is further arranged to receive an externalcomputing system identifier of one of the respondents, and identify oneor more of the other respondents based on the external computing systemidentifier of the respondent.

In an embodiment, system is further arranged to identify the otherrespondents by retrieving data enabling an identification of respondentsfrom an external computing system using the external computing systemidentifier of the respondent.

In an embodiment, system is further arranged to:

-   -   filter the time series of output data to generate a respondent        time series of output data set that excludes responses not        received from the identified other respondents;    -   generate message data based on the respondent time series of        output data set; and    -   control the communication interface to transmit the message data        to the external computing system using the external computing        system identifier of the respondent.

In an embodiment, the external computing system provides a socialnetworking website.

In a ninth aspect, the invention provides an electronic method forgenerating group output data for display, the method comprising:

-   -   obtaining, for each of a plurality of respondents, a time series        of response data corresponding to the same time-varying        stimulus;    -   receiving a user selection of one or more respondent        characteristics of a plurality of respondent characteristics        stored in association with one or more respondents in respective        respondent profiles stored in a respondent database;    -   identifying which of the plurality of respondents match the        selected one or more respondent characteristics by querying the        respondent database; and    -   generating a time series of group output data by combining the        respective time series of response data of the identified        respondents.

In an embodiment, the method comprises receiving an external computingsystem identifier of one of the respondents, and identifying one or moreof the other respondents based on the external computing systemidentifier of the respondent.

In an embodiment, identifying the other respondents comprises retrievingdata enabling an identification of respondents from an externalcomputing system using the external computing system identifier of therespondent.

In an embodiment, the method comprises:

-   -   filtering the time series of group output data to generate a        respondent time series of group output data set that excludes        responses not received from the identified other respondents;    -   generating message data based on the respondent time series of        group output data set; and    -   transmitting the message data to the external computing system        using the external computing system identifier of the        respondent.

In an embodiment, the external computing system provides a socialnetworking website.

In a tenth aspect, the invention provides an output data generationsystem, the system arranged to:

-   -   obtain, for each of a plurality of respondents, a time series of        response data corresponding to the same time-varying stimulus;    -   receive a user selection of one or more respondent        characteristics of a plurality of respondent characteristics        stored in association with one or more respondents in respective        respondent profiles stored in a respondent database;    -   identify which of the plurality of respondents match the        selected one or more respondent characteristics by querying the        respondent database; and    -   generate a time series of group output data by combining the        respective time series of response data of the identified        respondents.

In an embodiment, system is further arranged to receive an externalcomputing system identifier of one of the respondents, and identify oneor more of the other respondents based on the external computing systemidentifier of the respondent.

In an embodiment, system is further arranged to identify the otherrespondents by retrieving data enabling an identification of respondentsfrom an external computing system using the external computing systemidentifier of the respondent.

In an embodiment, system is further arranged to:

-   -   filter the time series of group output data to generate a        respondent time series of group output data set that excludes        responses not received from the identified other respondents;    -   generate message data based on the respondent time series of        group output data set; and    -   control the communication interface to transmit the message data        to the external computing system using the external computing        system identifier of the respondent.

In an embodiment, the external computing system provides a socialnetworking website.

In an eleventh aspect, the invention provides an electronic method forgenerating stimulus comparison data, the method comprising:

-   -   receiving a user selection of one or more stimulus        characteristics of a plurality of stimulus characteristics        stored in a database in association with sets of response data        of groups of respondents to respective ones of a plurality of        time-varying stimuli;    -   identifying which of the sets of response data match the        selected one or more stimulus characteristics by querying the        database; and    -   processing at least part of the response data to generate        stimulus comparison data indicative of a normal response of at        least a subset of respondents to the time-varying stimuli to        which the identified sets of response data belong.

In an embodiment, the method comprises generating a time series ofstimulus comparison data indicative of a time-varying normal response.

In an embodiment, the method comprises:

-   -   obtaining output data for a current time-varying stimulus; and    -   comparing the obtained output data with the stimulus comparison        data.

In an embodiment, the method comprises receiving an external computingsystem identifier of one of the respondents, and identifying one or moreof the other respondents based on the external computing systemidentifier of the respondent.

In an embodiment, identifying the other respondents comprises retrievingdata enabling an identification of respondents from an externalcomputing system using the external computing system identifier of therespondent.

In an embodiment, the method comprises:

-   -   filtering the stimulus comparison data to generate a respondent        stimulus comparison data set that excludes responses not        received from the identified other respondents;    -   generating message data based on the respondent stimulus        comparison data set; and    -   transmitting the message data to the external computing system        using the external computing system identifier of the        respondent.

In an embodiment, the external computing system provides a socialnetworking website.

In a twelfth aspect, the invention provides an output data generationsystem arranged to:

-   -   receive a user selection of one or more stimulus characteristics        of a plurality of stimulus characteristics stored in a database        in association with sets of response data of groups of        respondents to respective ones of a plurality of time-varying        stimuli;    -   identify which of the sets of response data match the selected        one or more stimulus characteristics by querying the database;        and    -   process at least part of the response data to generate stimulus        comparison data indicative of a normal response of at least a        subset of respondents to the time-varying stimuli to which the        identified sets of response data belong.

In an embodiment, the system is further arranged to generate a timeseries of stimulus comparison data indicative of a time-varying normalresponse.

In an embodiment, the system is further arranged to:

-   -   obtain output data for a current time-varying stimulus; and    -   compare the obtained output data with the stimulus comparison        data.

In an embodiment, the system is further arranged to receive an externalcomputing system identifier of one of the respondents, and identify oneor more of the other respondents based on the external computing systemidentifier of the respondent.

In an embodiment, the system is further arranged to identify the otherrespondents by retrieving data enabling an identification of respondentsfrom an external computing system using the external computing systemidentifier of the respondent.

In an embodiment, the system is further arranged to:

-   -   filter the stimulus comparison data to generate a respondent        stimulus comparison data set that excludes responses not        received from the identified other respondents;    -   generate message data based on the respondent stimulus        comparison data set; and    -   control the communication interface to transmit the message data        to the external computing system using the external computing        system identifier of the respondent.

In an embodiment, the external computing system provides a socialnetworking website.

The invention also extends to computer program code which when executedimplements one or more of the above methods. The computer program codemay be embodied on a tangible computer readable medium.

It should be noted that any of the various features of each of the aboveaspects of the invention, and of the various features of the embodimentsdescribed below, can be combined as suitable and desired.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described by way ofnon-limiting example in conjunction with the following drawings, inwhich:

FIG. 1 is a block diagram showing the components of an output generationsystem;

FIG. 2A is a block diagram showing the components of a mobile client;

FIG. 2B is a functional block diagram showing the functional componentsof the client of FIG. 2A;

FIG. 2C shows an example of a user interface for responding to atime-varying stimulus;

FIG. 3 is a block diagram of an alternative client;

FIG. 4 shows the structure of a session set of an embodiment;

FIG. 5 illustrates an alternative outward generation system;

FIG. 6 is a functional block diagram of a main server.

FIG. 7 is a functional block diagram of a live player server;

FIG. 8 is a flowchart of a method of an embodiment;

FIG. 9 is a flowchart of another method of an embodiment;

FIG. 10 is a flowchart of another method of an embodiment;

FIG. 11 is a flowchart of another method of an embodiment;

FIG. 12 is a functional block diagram of a playback server; and

FIG. 13 is a flowchart of another method of an embodiment.

DETAILED DESCRIPTION

Referring to the drawings, there are shown embodiments of an output datageneration system which can be used in electronic data generationsystems for generating output data corresponding to the responses ofrespondents to time-varying stimuli. Also illustrated are variouscomponents of the output data generation system which assist in thegeneration of data.

Embodiments of the invention find application in the monitoring of anaudience of respondents response to a time varying stimulus. The natureof the time-varying stimulus will vary from embodiment to embodiment.

A time-varying stimulus is a stimulus presented to a respondent forresponse over a period of time such that the stimulus to which therespondent responds changes and hence the respondent's response maychange. Examples of time varying stimuluses could include, live eventssuch as a debate, a play or a sporting event, transmissions of suchevents, transmissions of pre-recorded audio and/or visual programs etc.Depending on the embodiment, transmissions may be broadcast to pluralrespondents (and indeed to non-respondents at the same time ornarrowcast to a specific respondent (while the time of presentation ofthe stimulus is still controlled on the transmission side). In otherembodiments, the time of presentation of the stimulus may be controlledby the respondent.

As indicated above, responses to a time-varying stimulus may be gatheredin a number of different ways.

FIG. 1 shows an example of an output data generation system which isintended to be used to generate near real-time response data to a timevarying stimulus such as a broadcast event. For example, in order toobtain audience responses to a television program while it is beingbroadcast in order to generate response data that can be presented inconjunction with the television program. For example, as a graphicoverlay on display of the television program. In the embodiment, theprogram is broadcast using conventional means such that the respondentsview the program on a television. However, it will also be appreciatedthat the program could be broadcast using internet protocol or otherwisestreamed to a respondent device while enabling the respondent torespond.

The output generation system 100 comprises a number of server sidecomponents 110 which communicate over the Internet 130 with clientdevices (labelled as respondent devices to indicate that they belong torespondents). In the embodiment, the client side components willtypically be provided in the form of a computer program which can becaused to run on the relevant device in order to provide the specificfunctionality required to record response data and communicate it to theserver side components while leveraging off the standard componentsfound in the devices.

FIG. 1 shows a mobile respondent device 140A, which would typically be asmart phone, that is, a mobile telephone device with a sufficientlypowerful processor to run third party software. Also shown is a fixedrespondent device 140B which would typically be in the form of acomputer in data communication with the server side components 110 overa wide area communication network such as the Internet 130. A personskilled in the art will appreciate that the lines between mobile devicesand fixed devices can be somewhat blurred and herein, the distinction ismade merely to indicate that there are possible different types ofrespondent devices which can communicate, for example, over cellulardata communications networks or fixed line/hybrid fixed line wirelessnetworks. Persons skilled in art will also appreciate that there will bea plurality of respondent devices 140 but that only two are shown forsimplicity. In a further embodiment, a plurality of respondent devices161A-161N are connected to a concentrator 160, that aggregates data fromthe respective devices for communication over the network.

Respondents are enrolled to be members of an audience or panel in anumber of ways in respect of which group output data is to be generated.For example, the respondents may already be in a database owned by thecompany seeking to measure responses to the broadcast event, may bespecifically sourced by direct contact or advertisement, or may selfenroll in response to an advertisement or information or othercommunication of the existence of the possibility to be a respondent.Once a respondent enrolls, they download the client application ontotheir respondent device 140. In the following description, it is assumedthat the respondent has installed an operational client on theirrespondent device 140.

Also shown in FIG. 1 is a user terminal 120 to indicate schematicallythat a user may interact with various components of the server sidecomponents 110. Persons skilled in the art will appreciate that theremay be different terminals with different access rights to differentparts of the system and as such that a single user terminal 120 is shownto indicate the ability of users to interact with the system.

One aspect of the system 100 is the ability to schedule the plurality ofjobs in the system. This is done by a user configuring details of a jobin a job database 112. For example, the job may be to gather responsesto a talent show broadcast by a television station. The user may editthe job database 112 to specify the types of respondents who mayparticipate in providing the audience response to the talent show. Asthis is a broadcast television show, the person editing the job mayspecify that there are no specific requirements, however, they may beinterested in the different breakdowns (segmentations) of thedemographics of the audience response for men and women as well aspeople under and over the age of 40. When the television program is tobe broadcast, it is advertised—for example, during the first five minuteof the show—that live audience response be gathered and the job isloaded into the job administration server 111. Potential respondentsoperate their devices to indicate that they wish to participate as partof the audience. The devices 140 contact the job administration server111 based on a job identifier, for example, entered manually by therespondent. The job administration server 111 assigns an identificationfor this session to each respondent device 140 which is used incommunications between the devices to identify the respondent. The jobadministration server 111 also administers questions by communicatingwith the respondent device 140 to obtain information regarding therespondent's gender (i.e. male or female) and age in order to fulfil therequirements of this job for segmentation of the audience into differentportions of respondents via an input device.

In other embodiments, the respondent device 140 may communicate arespondent identifier to the job administration server 111 which maylook up the respondent's details in a respondent database 114 which havebeen captured at a previous time. If the job administration server 111determines that the respondent database 114 includes all the informationrequired to answer the questions, the job administration server 111 maynot ask the respondent any questions. Alternatively, the jobadministration server may ask any questions which are needed tosupplement the information in the respondent database 114. A furthertask of the job administration server 111 is to identify the main server113 with which the mobile respondent device should communicate while therespondent is responding. In this respect, it will be appreciated thatwhile one main server is shown in FIG. 1, there may be a plurality ofmain servers arranged in a number of different configurations. Forexample, one server or a group of servers may be allocating to aparticular job. If there are a group of servers, the servers may bearranged in a number of different ways such as in a two tierarrangement; a first tier handling direct communication with therespondent devices and a second tier processing the data from all of theservers in the first tier.

In a typical application, the television show will then indicate to therespondent when they should start responding. Further details of howresponses are handled are described below, however, in one embodimentthe respondents respond by moving a slider bar 212 relative to a scale211 displayed on display 210 of the respondent device 140A. The scalemay appear, for example, as shown in FIG. 2C.

Accordingly, it will be appreciated that the respondent's responseentered via the respondent device can also vary over time while thetime-varying stimulus is presented (in the form of a broadcasttelevision show in this example).

The main server 113 is arranged to communicate with the respondentdevices to receive communication of each respondent's response to thebroadcast transmission and to process these to generated output data. Inthe embodiments, this output data is further processed to generate groupdata output indicative a collective response of the audience ofrespondents (or a subset/segment thereof) to the show for display aspart of the broadcast transmission in conjunction with the display ofthe show. Accordingly, it will be appreciated that the talent showexample, whether the audience likes a particular performer or not can bedisplayed in conjunction with the display of the performer's performanceon the talent show as well as how the audience response to the performerchanges over time. Such an output can be generated in a number ofdifferent ways. In one example, data generated by the main server 113 isprovided to a live player 115 which formats the data into an appropriategraphical form in order for it to be output to a display system. Forexample, in a form that can be overlayed immediately over a televisionbroadcast. Alternatively, or in addition, the main server 113 may sendoutput data 118 to a television station (or an intermediary) such thatthey generate output graphics in a manner in which they desire and/orselect what data to use.

Also, in an alternative embodiment, data comprising or derived from thedata generated by the main server 113 may be sent to one or moreexternal computing systems. For example, in an embodiment, the outputdata generation system 100 may be connected to an external computingsystem providing a social networking website, and data comprising orderived from the data generated by the main server 113 may be sent tothe external computing system. Examples of social networking websitesinclude Facebook (Trade Mark), Twitter (Trade Mark), Linkedin (TradeMark) etc. In such an embodiment, the output data generation system 100may include a message data generator which formats the generated datainto format that can be posted on the social networking website providedby the external computing system. After the message data formats thegenerated data into a suitable format, the suitably formatted messagedata can be communicated from the main server 113 to the externalcomputing device. The message data can be communicated to the externalcomputing device using an external computing system identifier (such asa Facebook username and password) of one of the respondents receivedfrom one of the respondent devices. Persons skilled in the art willappreciate that the format of the message data generated by the messagedata generator may be different depending on which social networkingwebsite the message data is to be posted to.

In this respect, additional processing may be carried out on the datagenerated by the main server 113 based on the external computing systemidentifier of the respondent before the message data generator generatesthe message data. For example, the output data generation system 100 mayinclude an external computing system data retriever which retrievesexternal computing system data from the external computing system usingthe external computing system identifier of the respondent, and the datagenerated by the main server 113 may be filtered to exclude certainresponses based on the respondent data. For example, the externalcomputing system data may include data enabling an identification of oneor more of other respondents (such as a list of the respondent's friendson Facebook), and the data generated by the main server 113 may befiltered to exclude responses that are not from identified respondents.In another example, the external computing system data may includerespondent profile data (indicating, for example, that the respondentprefers to receive information displayed in the form of one or moremetrics, scores, or metrics and scores), and the data generated by themain server 113 may be processed to generate the message data accordingto the respondent profile data.

Persons skilled in the art will appreciate that the external computingsystem may provide a social networking service in a different form, forexample, in the form of a mobile application or mobile applicationsoftware (that is, an “app”).

Persons skilled in the art will also appreciate that the message datacommunicated to the external computing system may be transmitted to oneor more of the respondent devices. For example, after the message datais communicated to the external computing system, the message dataposted on the social networking website may be transmitted to arespondent device when the respondent device is used to view the socialnetworking website or websites. As indicated above, it will beappreciated that the message data transmitted to one respondent devicemay be different from the message data transmitted to another respondentdevice when the external computing device identifiers used by the tworespondent devices are different.

In addition, the main server 113 ensures that copies of the respondents'responses are stored for future use. In this respect, two copies of theeach respondents set of time series responses to the time bearingstimulus are kept. A first copy as a record against the respondent'sfile in the respondent database 114 and a second copy as part of asession data set of all responses for the session stored in a sessiondatabase 119 of a playback server 116. The playback server can be used alater date to retrieve a particular session and play it back. Further,as described in further detail below the playback server can be used tochange the segmentation of the responses by querying the respondentdatabase 114 based on respondent characteristics that may have not beenpart of the initial job specification.

FIG. 2A shows the typical components of a mobile respondent device 140A.As can be seen, the device 140A includes a display 210, at least oneinput device 220 such as a touch screen or keypad, a processor 230 andat least one transceiver circuit 250. A client program 241 is stored inmemory 240 such that when the processor executes the program 241, aclient 231 is instantiated on the device.

The client 231 is shown in FIG. 2B. The client 231 includes acommunication handler 270 arranged to establish communications with thejob administration server 111 and/or the main server 113 as needed. Theclient also includes a job enroller 280 which is arranged to initiallycontact the job administration server 111 based, for example, on a showidentifier and prompt the user via the display for them to input,responses to any questions that are received from the job administrationserver 111. The client also includes a display presenter 290 whichcontrols the display 210 to display the interface shown in FIG. 2C inthe form of a slider bar 212 over scale 211 such that the user can inputa response by moving the slider bar relative to the scale 211. Personsskilled in the art will appreciate that other scales can be used forexample, a scale from very negative to very positive.

Persons skilled in the art will appreciate that in other embodiments thedisplay presenter 290 may present additional material such as a portionof the transmission. It is advantageous if the input device 220 can be atouch screen overlaying the display such that the user can operate theslider bar 212 by operating the touch screen with their finger to dragthe slider bar left or right.

Persons skilled in the art will appreciate that other input techniquescould be used, for example, a slider bar could be moved operating apointing device such as a mouse or by using keys on a keyboard orkeypad. In other embodiments, the user interface may be different. Forexample, a graphical representation of a dial that a respondent rotatesto indicate their response.

The response data generator 260 includes a number of components that areused to improve the quality of captured response data.

In this respect, it will be appreciated that in embodiments where it isintended to produce a time-varying series of response data representingthe collective responses of respondents to particular portions of thetime-variance stimulus in near to real-time, it is important that theresponses that are assembled at the main server be related relevantly intime to the broadcast transmission. To this end, the client takesadvantage of the fact that most modern mobile telecommunication devicesstore the network time of the telecommunication network to which theyare connected which is usually highly accurate. Accordingly, as part ofa response assembler 264 assembling response data for communication themain server, a time stamp is added to each communication of responsedata to the main server. In this respect, the response data generatedincludes a response data recorder 262 which records the current positionof the slider bar 212 four times every second. The response assembler264 is also intended to communicate a current set of response data fourtimes a second. Accordingly, each time the response data recorderrecords a response, the response assembler obtains a time stamp fromtime stamper 263 and assembles a packet of response data 264 which itcommunicates to the main server 113.

Referring to FIG. 6, the main server 113 comprises a communicationinterface 610 for communicating with each respondent device 140.

The live data processor 640 is arranged to exclude data which has beenaffected by transmission lag in each time period. In this respect, itwill be appreciated that individual communications from individualrespondent devices can take longer than others to reach the main server113. If all of the responses were processed in the same way, responsesreceived at a particular time could relate to different portions of theprogram leading to an invalid output. Excessive communication delay fromindividual devices is advantageously eliminated by the live dataprocessor 640 filtering responses in each time epoch to exclude dataoutside of a timing tolerance. The timing tolerance can be set based onfactors such as the relative location of the server relative to thebroadcast or based on measured delivery to ensure that a sufficientnumber of respondents are captured. Accordingly, the live data processorincludes a filter 641 which include a time stamp checker 642 whichchecks the time stamp in each individual piece of response data receivedfrom the respondent devices 140 against local time at the main server113. In addition, the live data processor 640 includes a collectiveresponse assembler 643 which assembles data indicative of a collectiveresponse, for example to the average score of each included piece ofresponse data in the time epoch. The collective response assembler 643also generates collective responses corresponding to each segmentationof the data specified for this job based on respondent data 650. In thisexample based on gender (male/female) and age group under/over 40. Thiscan be then rendered into a display for display on the graphic as agraphic over the broadcast so that respondents and other viewers of thetelevision program can see how the audience is responding to thetelevision program.

In addition, a recorded data processor 630 records all data for theparticular session for saving to the respondent database and the sessiondatabase as described above. This is obtained by recording the data foreach respondent in conjunction with the respondent data 650 previouslypassed by the job administration server and main server 113. That is,each respondent's response that is provided is processed into a timeseries of response data items which are associated with respective onesof particular respondent. Individual respondent's data is stored againsttheir respondent records. The responses of all the respondents arestored together as the session data.

A data logger 620 keeps a copy of all communications in case data islost during processing and it is necessary to re-establish the session.

A further level of filtering the data is implemented at the client 231.In this respect the response data generator 260 includes an attentionmonitor 261. In the embodiment, the attention monitor 261 monitors theinput device in the form of a touch screen to determine whether anattention criterion was met. That is, the attention monitor whichmonitors the input device so that the response data recorder 262 onlyrecords data when a response criteria is met such that responses arethen only sent to the main server while a response criterion is beingmet. In the example of a touch screen, a response criterion may be thatthe user has maintained contact between their finger and the touchscreen and not released the touch screen. An alternative responsecriterion which may be used in relation to a touch screen or other inputdevices is that the user has moved the slider bar within a definedperiod and is hence actively changing the response that they areproviding in response to the stimulus rather than absorbing the materialpassively.

It will be appreciated that this has the advantage of excludingresponses of respondents who are not engaged with the stimulus. This isparticularly advantageous in embodiments where the response is remotelylocated such that it is not possible to supervise the respondents todetermine whether they are paying attention or not.

Processing other than filtering (as described above) may be implementedat the client 231. For example, the job enroller 280 may include amessage data generator arranged to, upon transmission of a jobidentifier to the job administration server 111, generate message dataand control the communication handler to transmit the message data to anexternal computing system (such as Facebook) using an external computingsystem identifier of the respondent operating a respondent device. Forexample, when the respondent device transmits a job identifier to thejob administration server 111, a message may be generated and posted onFacebook to alert Facebook friends of the respondent that the respondentis reacting to a particular time-varying stimulus. The message data mayinclude information indicative of, or related to, the time-varyingstimulus (for example, a description of the time-varying stimulus).Persons skilled in the art will appreciate that message data may begenerated and transmitted from the respondent device to more than oneexternal computing system. For example, separate message data may begenerated and transmitted to Facebook and Tweeter.

Also, it is envisaged that message data may be alternatively generatedon the server side of the output data generation system 100 and that themessage data may be communicated from the server side to the externalcomputing system or systems. For example, the main server 113 mayreceive external computing systems identifiers of respondents fromrespective respondent devices before a broadcast transmission, and themessage data may be communicated from the main server 113 to theexternal computing system or systems using the external computing systemidentifiers. The message data may be generated and transmitted inresponse to receipt of job identifiers from respondent devices.Alternatively, the message data may be generated and transmitted whenthe main server 113 receive responses from the respondents.

As indicated above, respondents could also respond to a time varyingstimulus in other circumstances. For example, narrowcast transmissionscould be made over the internet to respondent devices in the form ofcomputers of a selected group of respondents and those respondents couldrespond in real-time in a manner analogous to that described above.

FIG. 3 illustrates an alternative embodiment where respondents are notrequired to respond in real-time but rather can respond at a time oftheir choosing. In this case, the respondent device 331 has a differentconfiguration of the attention monitor 361 and display presenter 390. Inthis embodiment, the attention monitor 361 is arranged to monitor theinput device and pause presentation of the time-varying stimulus, if itdetects that the user is no longer paying attention. Presentation of thetime-varying stimulus will then resume once under control of displaypresenter 390 once the user is paying attention again such that anentire session of data will be generated by the response data generatedonce the user has monitored the entire stimulus.

FIG. 4 illustrates the typical components of a session set 400 whichcomprises a set of data which is presented to the user. The session set400 includes pre-test questions 402, a control element 412, a number oftest elements, in this example four test elements 422 a to 422 d, andpost test questions 404. The test elements are known as the test set.The control element is used to assess how the user responds.

In the embodiment, metadata 441-446 can be incorporated into the controlof test elements and/or at a specific point of interest 430 within aspecific test element. The metadata can then be used to generate outputat specific points, for example, in addition to displaying thecollective responses of respondents to the time varying stimulus. In theembodiment, the metadata is used to in a number of ways, including topopulate response data (e.g. with information identifying a controlelement). Another manner in which metadata may be used is to providedifferent segmentations of the output data depending on the metadata.For example, “Metadata 2” 442 may specify that a male/femalesegmentation is required such that average responses of male and femalerespondents are displayed whereas “Metadata 3” 443 may specify that anage group by age group output is required. Further, the metadata mayspecify when an output is displayed. For example, in addition to thetime-varying data, it may be desired to output metrics indicative ofother different measures such as likeability, peak scores, sustainlikability, critical likability, cut through and end score. “Metadata 6”446 can cause an output of a specific metric (e.g. likeability) at thepoint of interest 430. For example, a predefined high point in anadvertisement.

Persons skilled in the art will also appreciate use of metadata can, insome embodiments, enable the stimulus to be re-ordered as part of aprocess of randomising the stimulus presented to different respondents.The responses then can be automatically reassembled into responses tothe different segments. This can remove effects caused by the user'sattention varying over the entirety of the test set 420.

FIG. 7 shows the live player 115 which is arranged to generate graphicrepresenting a collective time varying response. The live player 115includes a data receiver 710 and a graphic generator 720. The liveplayer receives the collective response data from the live dataprocessor 640 and formats it into desired graphics. The output thentypically has on one axis, a time and on another axis measure of thecollective response based on a score, e.g. from 0 to 100 or +50 to −50or strongly liked to strongly disliked.

FIG. 5 shows an alternative output generation system that may be used,for example, in an application where users are seated in an audience towatch a debate. In this example, the output generation system 400comprises a number of client server side components 510 connected over alocal area network 520 to a plurality of identical user devices 540. Theuser devices may include, for example, a dial or a slider operable bythe respondents to respond the time-varying stimulus.

Referring again to FIG. 1, it is shown that the server side components110 include a playback server 116. The playback server 116 can be usedto review the results of a session at a later time and to play theseback to interested parties. Accordingly, while it is indicated above,the live portion of the live display of the collective response of thegroup of respondents may exclude certain data received outside of atiming tolerance whereas all data that is received can be saved forlater review such that the playback server has richer data. The playbackserver is also arranged to query the respondent database in order toidentify other characteristics of respondents for alternativesegmentations than those which are specified as part of a job.Accordingly, the playback server includes a search engine 210 as shownin FIG. 12 for enabling the playback server to locate respondents withspecific characteristics based on respondent database. For example, itmay be decided that other characteristics of respondents would be ofinterest than those selected for the job. The search engine allowsfurther searching of the respondent database to locate, where possible,respondents having the desired characteristics. In this respect, asshown in FIG. 12, the search engine 1210 will in this embodiment operateunder control of the session data segmenter 1220 which will thengenerate graphics for graphic generator 1250 for output.

A further aspect of the playback server 116 is the ability to generatehistorical data with historical generator 1230. For example, whenassessing the effectiveness of a beer advertisement it may be desired todetermine how people have responded in the past to beer advertisements.Accordingly, historical data generator uses search engine 1210 to pollprevious session data 119 for data flagged as being related to beeradvertisements to establish the normal response to a beer advertisement.This can then be displayed in conjunction with the segmented data forthe present session or fed to a comparator function 1240 which generatesdata for output which is indicative of how the responses differ, forexample, whether the response is more or less positive. In anotherexample, the norm can be used as a benchmark for later advertisements.The norm can be represented as a single metric (e.g. likeability) or asa time-varying response (e.g. likeability over time). Further, norms canbe presented for specific segments, such as male dog owners who usepublic transport and vote for a left wing party or female lizard ownerswho drive sports cars. Further, responses can also be segmented based onother factors, for example: time of day, relative positioning oftime-varying stimulus (e.g. within an advertisement break), length oftime-varying stimulus, category of stimulus, etc. That is, data isstored in session database to enable the playback server to segmentadvertisements to develop relevant norms. Such stimulus characteristicdata may be stored when the job is specified and/or added subsequentlyusing user terminal 120. Thus, it is possible by this method to generatecomparison data for different time varying stimuli and compare theresponses of respondents to prior responses to other time-varyingstimuli. This also enables the generation of data indicative of thedifference between a group response to a current time-varying stimulusand other time-varying stimuli.

Persons skilled in the art will appreciate that when the playback serveris processing session data, the time stamps described above allowsynchronisation of the responses to the test stimulus such that, forexample, the output data can be displayed along a time axiscorresponding to the time-varying stimulus. When the group response isto be viewed the time-varying stimulus can be displayed (where thestimulus is visual) and the time-varying group response to the timebearing stimulus can be displayed either immediately below orsuperimposed on the time-varying stimulus.

As responses are synchronised with the stimulus so that they may bereviewed over time, and responses to points of time within in thestimulus can be identified, the creation of significantly more granularunderstanding of individual and aggregate response is enabled.

Persons skilled in the art will also appreciate that the invention isembodied as a series of methods which can be implemented electronically.FIG. 8 shows one such method which involves receiving responses 810,filtering those responses to exclude responses outside of a timeintolerance 820, generating data from included responses 830 beforemoving to the next output time period 840 and beginning the filteringprocess again such that respondent devices may be included in the outputdata of some periods and not of others.

FIG. 9 shows a further method 900 of electronically receiving input 910at a respondent device, monitoring 920 the input device based on anattention criterion and only communicating response data meeting thecriterion 930.

FIG. 10 shows a further method 1000 which involves displaying stimuluswith associated metadata 1010, receiving responses 1020, and generatingoutput based on the response and the metadata 1030.

FIG. 11 shows a further method 1100 of embodiments of the inventionwhich involves obtaining a time series of response data for eachrespondent 1110, receiving user selection of respondent characteristics1120, identifying respondents based on the selected characteristics 1130in the respondent database 114 and generating output based on timeseries data of each identified respondent 1140.

FIG. 13 shows a further method 1300 of embodiments of the inventionwhich involves, receiving 1310 a user selection of time-varying stimuluscharacteristics, identifying 1320 sets of response data corresponding tothe user selection and processing 1330 at least part of the sets ofresponse data to generate stimulus comparison data.

Further aspects of the method will be apparent from the abovedescription of the system. It will be appreciated that at least part ofthe method will be implemented electronically, for example, digitally bya processor executing program code. In this respect, in the abovedescription certain steps are described as being carried out by aprocessor or computing equipment that incorporates one or moreprocessors such as a server, it will be appreciated that such steps willoften require a number of sub-steps to be carried out for the steps tobe implemented electronically, for example due to hardware orprogramming limitations. For example, to carry out a step such asevaluating, determining or selecting, a processor may need to computeseveral values and compare those values.

As indicated above, the method may be embodied in program code. Theprogram code could be supplied in a number of ways, for example on atangible computer readable storage medium, such as a disc or a memorydevice, e.g. an EEPROM, (for example, that could replace part of memory103) or as a data signal (for example, by transmitting it from aserver). Further different parts of the program code can be executed bydifferent devices, for example in a client server relationship. Personsskilled in the art, will appreciate that program code provides a seriesof instructions executable by the processor.

Herein the term “processor” is used to refer generically to any devicethat can process game instructions and may include: a microprocessor,microcontroller, programmable logic device or other computationaldevice, a general purpose computer (e.g. a PC) or a server. That is aprocessor may be provided by any suitable logic circuitry for receivinginputs, processing them in accordance with instructions stored in memoryand generating outputs (for example on the display). Such processors aresometimes also referred to as central processing units (CPUs). Mostprocessors are general purpose units, however, it is also know toprovide a specific purpose processor, for example, an applicationspecific integrated circuit (ASIC) or a field programmable gate array(FPGA).

It will be understood to persons skilled in the art of the inventionthat many modifications may be made without departing from the spiritand scope of the invention, in particular it will be apparent thatcertain features of embodiments of the invention can be employed to formfurther embodiments.

It is to be understood that, if any prior art is referred to herein,such reference does not constitute an admission that the prior art formsa part of the common general knowledge in the art in any country.

In the claims which follow and in the preceding description of theinvention, except where the context requires otherwise due to expresslanguage or necessary implication, the word “comprise” or variationssuch as “comprises” or “comprising” is used in an inclusive sense, i.e.to specify the presence of the stated features but not to preclude thepresence or addition of further features in various embodiments of theinvention.

The invention claimed is:
 1. An electronic method for generating groupdata indicative of the collective response of at least a subset ofrespondents who are members of an audience at one or more locations, thecollective response being to a plurality of portions of a time-varyingstimulus presented to the audience, the method comprising: receiving, byan output data generation system, responses to the time-varying stimulusfrom respective ones of a plurality of respondent devices, eachrespondent device being (i) in data communication with the output datageneration system and (ii) operable by a member of the audience toparticipate as a respondent, each response (i) including data indicativeof a response of a respondent associated with the respective respondentdevice to the time-varying stimulus and (ii) being entered by theassociated respondent using the respective respondent device; filtering,by the output data generation system, responses corresponding to aplurality of portions of the time-varying stimulus to which the groupdata is intended to relate, to generate, for each one of the portions ofthe time-varying stimulus, a filtered response data set that excludesany delayed one or ones of the responses to the portion of thetime-varying stimulus, the filtering being performed, for each portionof the time-varying stimulus, by excluding each delayed response that is(i) entered by a respondent using a respondent device within a timeperiod corresponding to the portion of the time-varying stimulus but(ii) received from the respondent device by the output data generationsystem outside of a timing tolerance relative to the time periodcorresponding to the portion of the time-varying stimulus due to atransmission lag; processing at least a subset of the filtered responsedata sets to generate the group data; receiving an external computingsystem identifier of one of the respondents, and identifying one or moreof the other respondents based on the external computing systemidentifier of the respondent; filtering the responses to generate arespondent response data set that excludes responses not received fromthe identified other respondents; generating message data based on therespondent response data set; and transmitting the message data to theexternal computing system using the external computing system identifierof the respondent, wherein the message data comprises one or moremetrics, scores, or metrics and scores.
 2. A method as claimed in claim1, comprising outputting the group data to a display generation systemarranged to process the group data to generate a display of group data.3. A method as claimed in claim 1, comprising processing the at least asubset of the filtered response data set to generate a display of thegroup data.
 4. A method as claimed in claim 1, wherein a first responsefrom at least one specific respondent device is included in at least onefiltered response data set and a second response from the at least onespecific respondent device is excluded from at least one filtered dataset.
 5. A method as claimed in claim 1, wherein the response dataincludes time data indicative of a time at which the response data wasgenerated and the time data is checked when filtering the response datato determine whether communication of the response data involved a delaygreater than the timing tolerance.
 6. A method as claimed in claim 1,wherein identifying the other respondents comprises retrieving dataenabling an identification of respondents from an external computingsystem using the external computing system identifier of the respondent.7. A method as claimed in claim 6, wherein the external computing systemprovides a social networking website.
 8. A non-transitory computerreadable medium comprising computer program code which when executedimplements the method of claim
 1. 9. An output data generation systemfor generating group data indicative of the collective response of atleast a subset of respondents who are members of an audience at one ormore locations, the collective response being to a plurality of portionsof a time-varying stimulus, the system being in data communication witha plurality of respondent devices, each respondent device being operableby a member of the audience to participate as a participant, the systemcomprising: a communication interface arranged to receive responses tothe time-varying stimulus from respective ones of the plurality ofrespondent devices, each response (i) including data indicative of aresponse of a respondent associated with the respective respondentdevice to the time-varying stimulus and (ii) being entered by theassociated respondent using the respective respondent device; a dataprocessor including a memory, the processor arranged to: filter responsedata corresponding to a plurality of portions of the time-varyingstimulus to which the group data is intended to relate, to generate, foreach one of the portions of the time-varying stimulus, a filteredresponse data set that excludes any delayed one or ones of the responsesto the portion of the time-varying stimulus, the filtering beingperformed, for each portion of the time-varying stimulus, by excludingeach delayed response that is (i) entered by a respondent using arespondent device within a time period corresponding to the portion ofthe time-varying stimulus but (ii) received from the respondent deviceby the output data generation system outside of a timing tolerancerelative to the time period corresponding to the portion of thetime-varying stimulus due to a transmission lag, process at least asubset of the filtered response data sets to generate the group data;receiving an external computing system identifier of one of therespondents, and identifying one or more of the other respondents basedon the external computing system identifier of the respondent; filteringthe responses to generate a respondent response data set that excludesresponses not received from the identified other respondents; generatingmessage data based on the respondent response data set; and transmittingthe message data to the external computing system using the externalcomputing system identifier of the respondent, wherein the message datacomprises one or more metrics, scores, or metrics and scores.
 10. Asystem as claimed in claim 9, wherein the communication interface isarranged to output the group data to a display generation systemarranged to process the group data to generate a display of group data.11. A system as claimed in claim 9, wherein the data processor isarranged to process the at least a subset of the filtered response dataset to generate a display of the group data.
 12. A system as claimed inclaim 9, wherein a first response from at least one specific respondentdevice is included in at least one filtered response data set and asecond response from the at least one specific respondent device isexcluded from at least one filtered data set.
 13. A system as claimed inclaim 9, wherein the response data includes time data indicative of atime at which the response data was generated and the time data ischecked when filtering the response data to determine whethercommunication of the response data involved a delay greater than thetiming tolerance.
 14. A system as claimed in claim 9, wherein the dataprocessor identifies the other respondents by controlling thecommunication interface to retrieve data enabling an identification ofrespondents from an external computing system using the externalcomputing system identifier of the respondent.
 15. A system as claimedin claim 14, wherein the external computing system provides a socialnetworking website.
 16. An electronic method for generating group dataindicative of the collective response of at least a subset ofrespondents who are members of an audience at one or more locations, thecollective response being to a plurality of portions of a time-varyingstimulus presented to the audience, the method comprising: receiving, byan output data generation system, responses to the time-varying stimulusfrom respective ones of a plurality of respondent devices, eachrespondent device being (i) in data communication with the output datageneration system over at least one of a fixed line network or awireless network and (ii) operable by a member of the audience toparticipate as a respondent, each response (i) including data indicativeof a response of a respondent associated with the respective respondentdevice to the time-varying stimulus and a time stamp, and (ii) beingentered by the associated respondent using the respective respondentdevice; filtering, by the output data generation system, responsescorresponding to a plurality of portions of the time-varying stimulus towhich the group data is intended to relate, to generate, for each one ofthe portions of the time-varying stimulus, a filtered response data setthat excludes any delayed one or ones of the responses to the portion ofthe time-varying stimulus, the filtering being performed, for eachportion of the time-varying stimulus by checking time stamps ofresponses to exclude each delayed response that is (i) entered by arespondent using a respondent device within a time period correspondingto the portion of the time-varying stimulus but (ii) received from therespondent device by the output data generation system outside of atiming tolerance relative to the time period corresponding to theportion of the time-varying stimulus due to a transmission lag that isidentified from the time stamp of the delayed response; processing atleast a subset of the filtered response data sets to generate the groupdata; receiving an external computing system identifier of one of therespondents, and identifying one or more of the other respondents basedon the external computing system identifier of the respondent; filteringthe responses to generate a respondent response data set that excludesresponses not received from the identified other respondents; generatingmessage data based on the respondent response data set; and transmittingthe message data to the external computing system using the externalcomputing system identifier of the respondent, wherein the message datacomprises one or more metrics, scores, or metrics and scores.